classdef basic_DS

%  basic_DS implements the basic functions of a
%  datasource (DS) object, namely, it takes binned data and labels, 
%  and through the get_data method, the object returns k 
%  leave-one-fold-out cross-validation splits of the data which can subsequently
%  be used to train and test a classifier.  The data in the population vectors is 
%  randomly selected from the larger binned data that is passed to the constructor
%  of this object.  This object can create both
%  pseduo-populations  (i.e., populations vector in which the recordings  were 
%  made on independent sessions but are treated as if they were recorded simultaneously)
%  and simultaneously populations in which neurons that were recorded together
%  always appear together in population vectors.  
%
% Like all DS objects, basic_DS implements the method get_data, which has 
% the following form:  
%
%  [XTr_all_time_cv YTr_all_cv XTe_all_time_cv YTe_all_cv ADDITIONAL_DATASOURCE_INFO] = get_data(ds)
%  where:
%    XTr_all_time_cv{iTime}{iCV} = [num_features x num_training_points] is a
%      cell array that has the training data for all times and cross-validation splits; 
%    YTr_all_cv{iTime} = [num_training_point x 1] are the training labels;
%    XTe_all_time_cv{iTime}{iCV} = [num_features x num_test_points] is a
%      cell array that has the test data for all times and cross-validation splits;   
%    YTe_all_cv{iTime} = [num_test_point x 1] has the test labels; and 
%    ADDITIONAL_DATASOURCE_INFO contains additional information, namely, for this object
%      it contains the field .curr_bootstrap_sites_to_use which lists which sites (e.g,. neurons)
%      were selected for the randomly split of that data that was just returned.
%
%  The constructor for this object has the form:
%   ds = basic_DS(the_data, the_labels, num_cv_splits), where:
%   the_data{iSite} = [num_trials x num_time_points] is a data matrix
%     for each site (e.g., neuron)       
%   the_labels{iSite} = [num_trials x 1] contains a vector of labels for each trial (for each site)
%   num_cv_splits = scalar, indicates how many cross-validation splits there should be.
%
%  The basic_DS also has the following properties that can be set:
%
%  1. create_simultaneously_recorded_populations (default = 0).  If the data from all sites
%      was recorded simultaneously, then setting this variable to 1 causes the 
%      function to return simultaneous populations rather than pseudo-populations 
%      (for this to work all sites in 'the_data' must have the trials in the same order).  
%      If this variable is set to 2, then the training set is pseudo-populations and the
%      test set is simultaneous populations.  This allows one to estimate I_diag (as
%      described by Averbeck, Latham and Pouget in 'Neural correlations, population coding
%      and computation', Nature Neuroscience, May, 2006).  I_diag is a measure that gives a 
%      sense of whether training on pseudo-populations leads to a the same decision rule as 
%      when training on simultaneous populations.  
%
%  2.  bootstrap_with_replacement (default = 0).  This variable specifies whether 
%        the sites should be sample with replacement - i.e., if the data is 
%        sampled with replacement, then some sites will be repeated within a single 
%        population vector.  This allows one to do a bootstrap estimate of variance
%        of the results if different sites from a larger population had been selected
%        while also ensuring that there is no overlapping data between the training
%        and test sets.  
%                    
%  3. num_times_to_repeat_each_label_per_cv_split (default = 1).  This variable 
%        specifies how many times each label should appear in each cross-validation split.
%        For example, if this value is set to k, this means that there will be k
%        population vectors from each class in each test set, and there will be 
%        k * (num_cv_splits - 1) population vectors for each class in each training set split.  
%     
%  4. label_numbers_to_use (default = [], which imples using all unique labels present
%       in the the_labels cell array).  This specifies which labels to use, of the unique labels
%       that are present in the the_labels cell array. If only a subset of labels are listed, 
%       then only population vectors that have the specified labels will be returned.  
%
%  5. num_bootstrap_sites (default = -1, which means use all sites).  This variable specifies
%        how many sites should be randomly selected each time the get_data method is called.
%        For example, suppose length(the_data) = n, and num_bootstrap_sites = k, then each
%        time get_data is called, k of the n sites would randomly be selected to be included
%        as features in the population vector.  
%      
%  6. sites_to_use (default = [], which means select features from all sites).  This
%       variable allows one to only choose features from the sites listed in this vector
%       (i.e., features will only be randomly selected from the sites listed in this vector).
%    
%  7. sites_to_exclude (default = [], which means do not exclude any sites).  This allows          
%       one to not select featuers from particular sites (i.e., features will NOT be
%       selected from the sites listed in this vector).
%
%  8. time_periods_to_get_data_from (default = [], which means create one feature
%       for all times that are present in the_data{iSite} matrix).  This variable 
%       can be set to a cell array that contains vectors that specify which time bins 
%       to use as features from the_data. For examples, if time_periods_to_get_data_from = {[2 3], [4 5], [10 11]}
%       then there will be three time periods for XTr_all_time_cv and  XTe_all_time_cv 
%       (i.e., length(XTr_all_time_cv) = 3), and the population vectors for the 
%       time period will have 2 * num_bootstrap_sites features, with the population 
%       vector for the first time period having data from each bootstrap site from times
%       2 and 3 in the_data{iSite} matrix, etc..  
%
%  9. use_random_subset_of_k_labels_each_time_data_is_retrieved (default = -1).  If this 
%       variable is set to k > 1, then a random subset of labels k labels will be chosen each 
%       time get_data is run (i.e., the diffferent bootstrap runs will use a different subset
%       of k labels each time the get_data method is called).  
%
%
%  This object also has two addition method which are:
%  
%  1.  the_properties = get_DS_properties(ds)
%       This method returns the main property values of the datasource.
%
%  2.  ds = set_specific_sites_to_use(ds, curr_bootstrap_sites_to_use)
%        This method causes the get_data to use specific sites rather than
%        choosing sites randomly.  This method should really only be used by 
%        other datasources that are extending the functionality of basic_DS.
%
%
%  (could also include code to turn stimultaneous population trainin data into pseudo-populations) 



%==========================================================================

%     This code is part of the Neural Decoding Toolbox.
%     Copyright (C) 2011 by Ethan Meyers (emeyers@mit.edu)
% 
%     This program is free software: you can redistribute it and/or modify
%     it under the terms of the GNU General Public License as published by
%     the Free Software Foundation, either version 3 of the License, or
%     (at your option) any later version.
% 
%     This program is distributed in the hope that it will be useful,
%     but WITHOUT ANY WARRANTY; without even the implied warranty of
%     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%     GNU General Public License for more details.
% 
%     You should have received a copy of the GNU General Public License
%     along with this program.  If not, see <http://www.gnu.org/licenses/>.
    
%==========================================================================    




  
properties 
    
    
    
    the_data                      %  a cell array that contains all the data binned data in the format the_data{iNeuron}[num_trials x num_time_bins]      
    the_labels                    %  a cell array that contains vectors of labels that specify what occurred during each trial for all neurons the_data cell array    
    num_cv_splits                 %  how many cross-validation splits there should be 
    
    num_times_to_repeat_each_label_per_cv_split = 1;     %  how many of each unique label should be in each CV block
    label_numbers_to_use = [];          %  which labels to use (i.e., can specify to only get trials when particular labels/conditions were present)
    bootstrap_with_replacement = 0;         %  specify whether to sample neurons with replacement - if they are sampled with replacement, then some features will be repeated within a single point data vector
    num_bootstrap_sites = -1;         %  how many sites should be used for each bootstrap iteration - must be less than length(the_data)
    
    create_simultaneously_recorded_populations = 0;   % to use pseudo-populations or simultaneous populations (2 => that the training set is pseudo and test is simultaneous)
      
    sites_to_use = [];                %  a list of indecies of which sites (neurons) to use in the the_data cell array 
    sites_to_exclude = [];            %  a list of features that should explicitly be excluded
    time_periods_to_get_data_from = [];  %  a cell array specifying containing vectors that specify which time bins to use from the_data 
                                         %       cross_validation_splits effectively reducing the numebr of neurons used.  It should be noted that when repleated neurons are used, all data duplication appears in the same CV trials
                                         %       so there is no contamination with having repeatd data in different CV trials
    
                                         
    % set these if you want the datasource to return different random selection of labels (and data from those trials) each time
    use_random_subset_of_k_labels_each_time_data_is_retrieved = -1;  % if this is set to k > 1, then a random subset of labels k labels will be chosen each bootstrap iteration 
 
    
    
    % getting rid of this - if I want to check if the results are significant, I really need to just shuffle the data once inititally and run the whole procedure rather than shuffle on each bootstrap run
    % randomly_shuffle_labels = 0;   % setting either of these to 1 should cause the decoding to be at chance levels
    % randomly_shuffle_data = 0;                                       
    
   
    
    
end


properties (GetAccess = 'public', SetAccess = 'private')
     curr_bootstrap_sites_to_use = [];  % This specified which features should be used on a given random bootstrap iteration.  
                                % This should really be randomly selected each time the generate_cv_data is called, but some rare cases
                                % it is useful to set to some specific features, so I am going to have a method that allows this 
                                % to be set by outside calls.  
end





methods

    
        %function ds = basic_DS
        %end
    
    
        % the constructor...
        function ds = basic_DS(the_data, the_labels, num_cv_splits)
            
            
            
            ds.the_data = the_data;
            ds.the_labels = the_labels;
            ds.num_cv_splits = num_cv_splits;           
            
            
            % initialize variables (these can be overwritten later)
            ds.sites_to_use = 1:length(the_data);
            ds.label_numbers_to_use = unique(ds.the_labels{1});   % use all the labels as the default value  (assuming that neuron 1 has all the labels shown - which might not be a foolproof assumption) 
            %ds.num_bootstrap_sites = length(ds.sites_to_use);


            % if the labels are in the wrong orientation, transpose them
            %if size(ds.label_numbers_to_use, 1) > 1
            %    ds.label_numbers_to_use = ds.label_numbers_to_use';  
            %end
            
                        
            for i = 1:size(ds.the_data{1}, 2)    
                time_periods_to_use{i} = i;                
            end
            ds.time_periods_to_get_data_from = time_periods_to_use;  %ds = set(ds, 'time_periods_to_get_data_from', time_periods_to_use);            
                  
            
            
        end
        
        
        
        
        function ds = set.create_simultaneously_recorded_populations(ds, create_simultaneously_recorded_populations)
           
            ds.create_simultaneously_recorded_populations = create_simultaneously_recorded_populations;
            
            % convert data back and forth between simultaneous data matrix format and psuedo-population cell array format
            

            if create_simultaneously_recorded_populations == 0
                
                if ~(iscell(ds.the_data))
            
                    the_data = ds.the_data;
                    the_labels = ds.the_labels;
                    
                    for iSite = 1:size(the_data, 2)                       
                        curr_data{iSite} = squeeze(the_data(:, iSite, :));
                        curr_labels{iSite} = ds.the_labels;
                    end
                    
                    ds.the_data = curr_data;
                    ds.the_labels = curr_labels;
                    
                end
                
            elseif create_simultaneously_recorded_populations > 0
               
                if iscell(ds.the_data)
            
                    simultaneous_labels_to_use = ds.the_labels{1};
                    the_data = ds.the_data;
                    
                    for iSite = 1:length(the_data)                        
                        
                            if sum(abs(simultaneous_labels_to_use - ds.the_labels{iSite})) ~= 0
                                error('problem, all simultaneously recorded neurons should have the same labels')   
                            end
                            
                            the_simultaneous_data(:, :, iSite) = the_data{iSite};                          
                    end
                    
                    ds.the_data = permute(the_simultaneous_data, [1 3 2]);
                    ds.the_labels = simultaneous_labels_to_use;
                end
                            
            end
                
        end
        
        
        
        
        % This method allows one to always use a prespecified set of sites 
        %  rather than randomly selecting a set of sites from the larger population (it should rarely be used)
        function ds = set_specific_sites_to_use(ds, curr_bootstrap_sites_to_use)
            ds.curr_bootstrap_sites_to_use = curr_bootstrap_sites_to_use;
        end
           
        
        %  This method returns the main property values of the datasource (not sure if this is really needed) 
        function the_properties = get_DS_properties(ds)
            
            the_properties.num_cv_splits =  ds.num_cv_splits; 
            the_properties.num_times_to_repeat_each_label_per_cv_split = ds.num_times_to_repeat_each_label_per_cv_split;
            the_properties.label_numbers_to_use = ds.label_numbers_to_use;
            the_properties.bootstrap_with_replacement = ds.bootstrap_with_replacement;
            the_properties.num_bootstrap_sites = ds.num_bootstrap_sites;
            the_properties.create_simultaneously_recorded_populations = ds.create_simultaneously_recorded_populations;
            the_properties.sites_to_use = ds.sites_to_use;
            the_properties.sites_to_exclude = ds.sites_to_exclude;
            the_properties.time_periods_to_get_data_from = ds.time_periods_to_get_data_from;                         
            the_properties.use_random_subset_of_k_labels_each_time_data_is_retrieved = ds.use_random_subset_of_k_labels_each_time_data_is_retrieved;
        
        end
        
        
        
        function  [XTr_all_time_cv YTr_all_cv XTe_all_time_cv YTe_all_cv ADDITIONAL_DATASOURCE_INFO] = get_data(ds) 
        % The main DS function that returns training and test population vectors.  The outputs of this function are:
        %    
        %  1. XTr_all_time_cv{iTime}{iCV} = [num_features x num_training_points] is a
        %        cell array that has the training data for all times and cross-validation splits; 
        %
        %  2. YTr_all_cv{iTime} = [num_training_point x 1] has the training labels;
        %
        %  3. XTe_all_time_cv{iTime}{iCV} = [num_features x num_test_points] is a
        %       cell array that has the test data for all times and cross-validation splits;   
        %
        %  4. YTe_all_cv{iTime} = [num_test_point x 1] has the test labels
        % 
        %  5. ADDITIONAL_DATASOURCE_INFO contains additional information, namely, for this object
        %      it contains the field .curr_bootstrap_sites_to_use which lists which sites (e.g,. neurons)
        %      where selected for the randomly split of that data that was just returned.


        
            % access to objects' fields in matlab is very slow (which is super pathetic), so to get the code to run faster I have use temporary copies of the data
            %  (hopefully matlab will fix this in the future)
            the_data = ds.the_data;                                                 
            the_labels = ds.the_labels;            
            num_cv_splits = ds.num_cv_splits;            
            num_times_to_repeat_each_label_per_cv_split =  ds.num_times_to_repeat_each_label_per_cv_split;                        
            curr_bootstrap_sites_to_use = ds.curr_bootstrap_sites_to_use;
            label_numbers_to_use = ds.label_numbers_to_use;  if size(label_numbers_to_use, 1) ~= 1, label_numbers_to_use = label_numbers_to_use'; end  % make sure labels numbers are in the correct orientation
            sites_to_use = ds.sites_to_use;
            sites_to_exclude = ds.sites_to_exclude;
            num_bootstrap_sites = ds.num_bootstrap_sites;
            use_random_subset_of_k_labels_each_time_data_is_retrieved =ds.use_random_subset_of_k_labels_each_time_data_is_retrieved;
            bootstrap_with_replacement = ds.bootstrap_with_replacement;   
            create_simultaneously_recorded_populations = ds.create_simultaneously_recorded_populations;
            

            % some santy checks...
            if isempty(sites_to_use) || (isscalar(sites_to_use) &&  sites_to_use == -1)
                sites_to_use = 1:length(the_data);
            end
        
            if ~isempty(sites_to_exclude)      % using custom neurons (i.e, excluding certain neurons b/c of trends, don't have enough repetitions, etc..)
                sites_to_use = setdiff(sites_to_use, sites_to_exclude);
            end
            
            if isempty(label_numbers_to_use) ||  (isscalar(label_numbers_to_use) && label_numbers_to_use < 1)
               label_numbers_to_use = unique(the_labels{1});   % use all the labels as the default value  (assuming that neuron 1 has all the labels shown - could be a bit screwy) 
            end
 
            if length(label_numbers_to_use) ~= length(unique(label_numbers_to_use))
               warning('some labels were listed twice in the field ds.label_numbers_to_use, (these duplicate enteries will be ignored)');
               label_numbers_to_use = unique(label_numbers_to_use);
            end
            
            
            
            % if the number of bootstrap neurons is not specified, use all neurons
            if num_bootstrap_sites < 1 || isempty(num_bootstrap_sites)
                num_bootstrap_sites = length(sites_to_use);
            end
            

            % code for randomly selecting k labels to use each time data is retrieved 
            if use_random_subset_of_k_labels_each_time_data_is_retrieved > 1   % needs at least 2 labels for a classification problem to work
                   rand_label = label_numbers_to_use(randperm(length(label_numbers_to_use)));
                   label_numbers_to_use = rand_label(1:use_random_subset_of_k_labels_each_time_data_is_retrieved); 
            end
            
            
            
            % if specific sites to be used have not been given (as should usually be the case), randomly select sites to use
            if isempty(curr_bootstrap_sites_to_use) 

                if ~(bootstrap_with_replacement)   % only use each feature once in a population vector
                    curr_bootstrap_sites_to_use = sites_to_use(randperm(length(sites_to_use)));
                    curr_bootstrap_sites_to_use = sort(curr_bootstrap_sites_to_use(1:num_bootstrap_sites));  % sorting just for the heck of it

                else   % selecting random features with replacement (i.e., the same feature can be repeated multiple times in a population vector).    
                    initial_inds = ceil(rand(1, num_bootstrap_sites) * num_bootstrap_sites);  % can have multiple copies of the same feature within a population vector
                    curr_bootstrap_sites_to_use = sort(sites_to_use(initial_inds));    
                end

            end

            
            % making code more robust in case transpose of curr_bootstrap_sites_to_use is actually passed as an argument
            if (size(curr_bootstrap_sites_to_use, 1) > 1)
                curr_bootstrap_sites_to_use = curr_bootstrap_sites_to_use';
            end
                  
           
           % pre-allocate memory... 
           all_data_point_labels = NaN .* ones(length(unique(label_numbers_to_use)) * num_cv_splits * num_times_to_repeat_each_label_per_cv_split, 1);           
           start_boostrap_ind = 1;  
            
   
           if  create_simultaneously_recorded_populations == 0

       
               % pre-allocate memory... 
                the_bootstrap_data = NaN .* ones(length(unique(label_numbers_to_use)) * num_cv_splits * num_times_to_repeat_each_label_per_cv_split, length(curr_bootstrap_sites_to_use), size(the_data{1}, 2)); 
          

                % if someone has changed the data or the labels after they have already set create_simultaneously_recorded_populations = 0, then the format of these variables needs to be converted back
                if ~(iscell(ds.the_data)) || ~(iscell(the_labels)) 
                    create_simultaneously_recorded_populations = 0;
                end
                
                
                
               % make sure label_numbers_to_use is a row vector
               if size(label_numbers_to_use, 1) > 1
                   label_numbers_to_use = label_numbers_to_use';
               end
                
                
               % create a 3 dimensional tensor the_bootstrap_data that is [(num_labels * num_cv_slits * num_repeats_per_cv_label)   x num_neurons x num_time_bins] large               
               for iLabel = label_numbers_to_use 

                    cNeuron = 1;    
                    for iNeuron = unique(curr_bootstrap_sites_to_use)  

                        % choose random trials to use for each label type
                        curr_trials_to_use = find(the_labels{iNeuron} == iLabel);  %find(ds.the_labels{iNeuron} == iLabel);
                        curr_trials_to_use = curr_trials_to_use(randperm(length(curr_trials_to_use)));

                        if length(curr_trials_to_use) < (num_cv_splits  * num_times_to_repeat_each_label_per_cv_split)    
                            error('problems:  asking for more trials of a given stimuli type then were recorded in the experiment');  % maybe this should be an error
                            return;
                        else
                            curr_trials_to_use = curr_trials_to_use(1:(num_cv_splits  * num_times_to_repeat_each_label_per_cv_split));
                        end


                        % put everything into the correct number of CV splits...
                        for iRepeats = 1:length(find(curr_bootstrap_sites_to_use == iNeuron))                    
                            the_bootstrap_data(start_boostrap_ind:(start_boostrap_ind + length(curr_trials_to_use) - 1), cNeuron, :) = the_data{iNeuron}(curr_trials_to_use, :);
                            cNeuron = cNeuron + 1;
                        end

                    end    % end iNeuron


                    all_data_point_labels(start_boostrap_ind:(start_boostrap_ind + length(curr_trials_to_use) - 1)) = iLabel .* ones(length(start_boostrap_ind:(start_boostrap_ind + length(curr_trials_to_use) - 1)), 1);

                    start_boostrap_ind = start_boostrap_ind + length(curr_trials_to_use);


                end   % end for iLabel  

                
                
            
            % if creating simultaneously recorded populations ...
           elseif  create_simultaneously_recorded_populations > 0
            
                               
               the_bootstrap_data = NaN .* ones(length(unique(label_numbers_to_use)) * num_cv_splits * num_times_to_repeat_each_label_per_cv_split, length(curr_bootstrap_sites_to_use), size(the_data, 3)); 
 
               the_data = the_data(:, curr_bootstrap_sites_to_use, :);

                % choose (num_cv * num_repeats) random data points for each class            
                for iLabel = label_numbers_to_use

                    % choose random trials to use for each label type
                    curr_trials_to_use = find(the_labels == iLabel);
                    curr_trials_to_use = curr_trials_to_use(randperm(length(curr_trials_to_use)));

                    if length(curr_trials_to_use) < (num_cv_splits * num_times_to_repeat_each_label_per_cv_split)
                        error('problems:  asking for more trials of a given stimuli type then were recorded in the experiment');  % maybe this should be an error
                        return;
                    else
                        curr_trials_to_use = curr_trials_to_use(1:(num_cv_splits * num_times_to_repeat_each_label_per_cv_split));
                    end

                    the_bootstrap_data(start_boostrap_ind:(start_boostrap_ind + length(curr_trials_to_use) - 1), :, :) = the_data(curr_trials_to_use, :, :);                
                    all_data_point_labels(start_boostrap_ind:(start_boostrap_ind + length(curr_trials_to_use) - 1)) = iLabel .* ones(length(start_boostrap_ind:(start_boostrap_ind + length(curr_trials_to_use) - 1)), 1);             
                    start_boostrap_ind = start_boostrap_ind + length(curr_trials_to_use);

                end
            
           end % end simultaneous populations
            
            


            clear the_data   % clear up some memory
            
        
            all_bootstrap_data_inds = 1:size(the_bootstrap_data, 1);
            
            time_periods_to_get_data_from = ds.time_periods_to_get_data_from;
            
            
            % convert the_bootstrap_data into a training and splits ...
            for iTimePeriod = 1:length(time_periods_to_get_data_from)
        
                curr_data = the_bootstrap_data(:, :, time_periods_to_get_data_from{iTimePeriod});            
                the_bootstrap_data_time = reshape(curr_data, [size(curr_data, 1) size(curr_data, 2) * size(curr_data, 3)]);

                
                if (create_simultaneously_recorded_populations > 1)  
                    the_site_ids = 1:size(curr_data, 2);
                    simul_to_pseudo_feature_to_siteID_mapping{iTimePeriod} = repmat(the_site_ids, [1 size(curr_data, 3)]);
                end
                
                
                cv_start_ind = 1;
                
                for iCV = 1:num_cv_splits
                   
                    curr_cv_inds = [];  %NaN .* ones(num_times_to_repeat_each_label_per_cv_split * length(ds.label_numbers_to_use), 1);
                    
                    for iNumRepeatsPerLabel = 1:num_times_to_repeat_each_label_per_cv_split
                        
                        curr_cv_inds  = [curr_cv_inds cv_start_ind:(num_cv_splits  * num_times_to_repeat_each_label_per_cv_split):size(the_bootstrap_data, 1)];
                        cv_start_ind = cv_start_ind + 1;
                    end
                    
                    
                    
                    % these cells arrays contain the data for each CV splits separately,
                    % but don't need to create these, rather this function will just return the CV data divided into training and test sets
                    %    % % cross_validation_splits_all_time_periods{iCV}{iTimePeriod} = the_bootstrap_data_time(curr_cv_inds, :);  % old get_bootstrap_data7 format...
                    %    cross_validation_splits_all_time_periods{iTimePeriod}{iCV} = the_bootstrap_data_time(curr_cv_inds, :)';  % should replace above with this soon
                    %    cross_validation_labels{iCV} = all_data_point_labels(curr_cv_inds);   % not sure why I need a separate one for each CV?

                   
                    % can actually just return these instead...
                    XTr_all_time_cv{iTimePeriod}{iCV} = the_bootstrap_data_time(setdiff(all_bootstrap_data_inds, curr_cv_inds), :)';  
                    XTe_all_time_cv{iTimePeriod}{iCV} = the_bootstrap_data_time(curr_cv_inds, :)';
                    
                    YTr_all_cv{iCV} = all_data_point_labels(setdiff(all_bootstrap_data_inds, curr_cv_inds));
                    YTe_all_cv{iCV} = all_data_point_labels(curr_cv_inds);
                    
                                 
                end                
                
            end
            
                       
            % save the inds of the random neurons chosen
            ADDITIONAL_DATASOURCE_INFO.curr_bootstrap_sites_to_use = curr_bootstrap_sites_to_use;


            % If create_simultaneously_recorded_populations == 2, create pseudo-populations for training, and simultaneous data for testing.
            % This is useful for assessing I_diag as described by Averbeck, Latham and Pouget, Nature Neurosience, May 2006.
            if  create_simultaneously_recorded_populations == 2
                XTr_all_time_cv = turn_training_simultaneous_data_into_pseudo_populations(XTr_all_time_cv, YTr_all_cv, simul_to_pseudo_feature_to_siteID_mapping);
            end
            
            % There really is no reason to use this, so I am not going to give this as an option for now...
            %if  create_simultaneously_recorded_populations == 3
            %    [XTr_all_time_cv XTe_all_time_cv] = turn_all_simultaneous_data_into_pseudo_populations(XTr_all_time_cv, YTr_all_cv, XTe_all_time_cv, YTe_all_cv, simul_to_pseudo_feature_to_siteID_mapping)
            %end
            if (create_simultaneously_recorded_populations  > 2) || (create_simultaneously_recorded_populations  <  0)
                warning('create_simultaneously_recorded_populations must be set to 0, 1 or 2');   % should really be an error not a warning, and should have this earlier in the code, but ok for now
            end
            
            
            %keyboard
            
        end   % end get_data
        
        
       
   end  % end methods
   
    
    


end % end class



